Electronic apparatus and image correction method thereof

ABSTRACT

An electronic device is provided. The electronic device includes a camera, a communication interface, and a processor configured to capture an image of a background of a display device through the camera, to divide the captured image into a plurality of blocks, to compare a target gradation value with gradation values of the each of the plurality of blocks, to adjust the gradation values of the each of the plurality of blocks, and to transmit an image of which the gradation values are adjusted, to the display device through the communication interface, wherein the sizes of the plurality of blocks are identified based on a background pattern of the display device.

TECHNICAL FIELD

The disclosure relates to an electronic device and an image correctionmethod thereof, and more particularly, to an electronic device whichcorrects an image and an image correction method thereof.

BACKGROUND ART

A display device such as a television, or the like is a device fordisplaying an image. Recently, not only a function for displaying animage, but also a function for providing a variety of experiences to auser has been added.

However, when the display device displays an image, when light isemitted by an ambient light source, the area where the light isilluminated gets brighter than other areas, so that the correspondingpart is recognized as a different color to the user.

DETAILED DESCRIPTION OF THE INVENTION Technical Problem

Accordingly, an object of the disclosure is to provide an electronicdevice capable of correcting a gradation value, for each block, of abackground image displayed on a display device using the electronicdevice, and an image correction method thereof.

Technical Solution

An aspect of the embodiments relates to an electronic device including acamera, a communication interface, and a processor configured to capturea background of a display device through the camera, to divide thecaptured image into a plurality of blocks, to compare a target gradationvalue with gradation values of the each of the plurality of blocks, toadjust the gradation values of the each of the plurality of blocks, andto transmit an image of which the gradation values are adjusted, to thedisplay device through the communication interface, wherein the sizes ofthe plurality of blocks are identified based on a background pattern ofthe display device.

The processor may be configured to identify the sizes of the blocks onthe basis of at least one of a size, complexity, and a type of thebackground pattern and to divide the captured image into the pluralityof blocks on the basis of the identified size.

The processor may be configured to divide the captured image into theplurality of blocks such that each block has a size corresponding to thebackground pattern, and the larger the size of the background patternis, the relatively larger the size of the block may become, and thesmaller the size of the background pattern is, the relatively smallerthe size of the block may become.

The processor may be configured, on the basis of frequency components ofthe captured image, to identify that the background pattern has a firstsize, based on high-frequency components being relatively more thanlow-frequency components in the captured image, and to identify that thebackground pattern has a second size, based on low-frequency componentsbeing relatively more than high-frequency components in the captureimage, wherein the second size is larger than the first size.

The processor may be configured to divide the captured image into theplurality of blocks such that each block has a size corresponding tocomplexity of the background, and the higher the complexity of thebackground is, the relatively larger the size of the block may become,and the lower the complexity of the background is, the relativelysmaller the size of the block may become.

The processor may be configured to calculate a variance of the capturedimage and divide the capture image into the plurality of blocks on thebasis of complexity corresponding to the variance among a plurality ofcomplexity.

The processor may be configured to divide the captured image into theplurality of blocks such that each block has a size corresponding to thebackground pattern.

The processor may be configured to identify a type of the backgroundpattern by performing image search for the captured image through a webserver.

The processor may be configured to normalize the gradation values of theeach of the plurality of blocks, compare the normalized respectivegradation values and average values of the normalized gradation valuesto calculate adjustment values of the each of the plurality of blocks,and adjust the gradation values of the each of the plurality of blockson the basis of the calculated adjustment values.

An aspect of embodiments relates to an image correction method of anelectronic device includes capturing a background of a display device,dividing the captured image into a plurality of blocks, comparing atarget gradation value with gradation values of the each of theplurality of blocks and adjusting the gradation values of the each ofthe plurality of blocks, and transmitting an image of which thegradation values are adjusted, to the display device, wherein the sizesof the plurality of blocks are identified based on a background patternof the display device.

The dividing may include identifying the sizes of the blocks on thebasis of at least one of a size, complexity, and a type of thebackground pattern and dividing the captured image into the a pluralityof blocks on the basis of the identified size.

The dividing may include dividing the captured image into the pluralityof blocks such that each block has a size corresponding to thebackground pattern, and the larger the size of the background patternis, the relatively larger the size of the block may become, and thesmaller the size of the background pattern is, the relatively smallerthe size of the block may become.

The dividing may include, on the basis of frequency components of thecaptured image, identifying that the background pattern has a firstsize, based on high-frequency components being relatively more thanlow-frequency components in the captured image, and identifying that thebackground pattern has a second size, based on low-frequency componentsbeing relatively more than high-frequency components in the capturedimage, wherein the second size is larger than the first size.

The dividing may include dividing the captured image into the pluralityof blocks such that each block has a size corresponding to complexity ofthe background, and the higher the complexity of the background is, therelatively larger the size of the block may become, and the lower thecomplexity of the background is, the relatively smaller the size of theblock may become.

The dividing may include calculating a variance of the captured imageand dividing the captured image into the plurality of blocks on thebasis of complexity corresponding to the variance among a plurality ofcomplexity.

Effect of the Invention

According to the various embodiments of the disclosure, a gradationvalue of an image can be corrected by each block having a sizeidentified based on a background pattern, so that a white balance of acaptured image can be finely corrected even in illumination environmentin which a plurality of light sources exist, and an influence of thepattern can be minimized when light shines on.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an operation of a display deviceaccording to an embodiment;

FIG. 2 is a block diagram illustrating a configuration of an electronicdevice according to an embodiment;

FIGS. 3 to 7 are diagrams illustrating a method of dividing an imageinto a plurality of blocks according to various embodiments;

FIG. 8 is a block diagram illustrating a detailed configuration of anelectronic device according to an embodiment;

FIG. 9 is a block diagram illustrating a configuration of a displaydevice according to an embodiment; and

FIG. 10 is a flowchart illustrating an image correction method accordingto an embodiment.

BEST MODE FOR IMPLEMENTING THE DISCLOSURE Mode for Implementing theDisclosure

The present disclosure may have several embodiments, and the embodimentsmay be modified variously. In the following description, specificembodiments are provided with accompanying drawings and detaileddescriptions thereof. However, this does not necessarily limit the scopeof the exemplary embodiments to a specific embodiment form. Instead,modifications, equivalents and replacements included in the disclosedconcept and technical scope of this specification may be employed. Whiledescribing exemplary embodiments, if it is determined that the specificdescription regarding a known technology obscures the gist of thedisclosure, the specific description is omitted.

The terms such as “first,” “second,” and so on may be used to describe avariety of elements, but the elements should not be limited by theseterms. The terms used herein are solely intended to explain specificexample embodiments, and not to limit the scope of the presentdisclosure.

The terms used herein are solely intended to explain a specificexemplary embodiment, and not to limit the scope of the presentdisclosure. Singular forms are intended to include plural forms unlessthe context clearly indicates otherwise. The terms “include”,“comprise”, “is configured to,” etc., of the description are used toindicate that there are features, numbers, steps, operations, elements,parts or combination thereof, and they should not exclude thepossibilities of combination or addition of one or more features,numbers, steps, operations, elements, parts or a combination thereof.

In the embodiments disclosed herein, a term ‘module’ or ‘unit’ refers toan element that performs at least one function or operation. The‘module’ or ‘unit’ may be realized as hardware, software, orcombinations thereof. In addition, a plurality of ‘modules’ or aplurality of ‘units’ may be integrated into at least one module and maybe at least one processor except for ‘modules’ or ‘units’ that should berealized in a specific hardware.

Below, example embodiments will be described in detail with reference tothe attached drawings.

FIG. 1 is a diagram illustrating an operation of a display deviceaccording to an embodiment.

The display device 200 according to an embodiment of the disclosureprovides two operation modes.

Here, the display device 200 may be implemented as a television.

Firstly, a first operation mode, for example, is a mode for displaying anormal image. Specifically, the first operation mode is a mode fordisplaying a pre-stored content or broadcast received from an externalsource by using an entire screen of the display device.

In addition, a second operation mode is a mode in which the displaydevice 200 displays a background screen so that the user may not easilyrecognize the display device. The background screen is a screen in whichthe user captured a background where the display device 200 is locatedor surroundings thereof in advance. In other words, the user may capturea background of a location where the display device 200 will be locatedbefore the display device 200 is located or a surrounding background ofthe display device 200 after the display device 200 is located throughthe electronic device 100. In addition, the electronic device 100 maytransmit the captured background image, and the display device 200 maydisplay the background image received from the electronic device 100.

When the display device is displayed in the second operation mode asdescribed above, the display device 200 may display a background areabehind the display device as a background image, and thereby the usermay feel that the display device has become a transparent glass window.

Meanwhile, in the second operation mode, the display device may displaya specific graphic object as well as the background screen. In thisregard, the specific object may be a clock object, but various objects(e.g., pictures, photos, fish tanks, etc.) that may be attached to anormal wall, may be displayed as well.

The electronic device 100 may capture a background of the display device200 and transmit the captured image to the display device 200.

In this regard, the electronic device 100 may be implemented as asmartphone. However, it is only an embodiment, and a terminal device 100may be implemented as various types of electronic devices, which areportable devices, such as a tablet personal computer (PC), a mobilephone, a personal digital assistant (PDA), a portable multimedia player(PMP), a wearable device, and the like.

In this regard, the electronic device 100 may correct a white balance ofthe captured image and transmit the corrected image to the displaydevice 200.

For example, when natural light such as the sun or room light shines onthe display device 200, a part of the image displayed by the displaydevice 200 may be bright, and other parts may be darkened according to alocation of the screen of the display device 200 where the light shines,and thus a color of the image may be seen different to the user.

Specifically, when the display device 200 displays the background screento give a transparent effect to the user, if an image displayed by thedisplay device 200 is shown in different colors to the user according tolight, the user may feel may feel difference between the image displayedon the display device 200 and the background, thereby reducing atransparent effect.

Accordingly, according to an embodiment of the disclosure, theelectronic device 100 may adjust gradation values of the captured imageto correct white balance of the image, and transmit the corrected imageto the display device 200. In this case, the electronic device 100 maydivide the captured image into a plurality of blocks, and correct thewhite balance of the captured image by adjusting the gradation values ofthe each of the plurality of blocks.

As described above, since the electronic device 100 corrects the whitebalance of the captured image by block, the electronic device 100 mayfinely correct the white balance of the captured image, even in theillumination environment in which light from a plurality of lightsources, such as natural light and room lighting, a plurality of indoorlights, or natural light and a plurality of indoor lights, or the like,shines on the display device 200, thereby resolving heterogeneity thatuser feels and maximizing a transparent effect.

FIG. 2 is a block diagram illustrating a configuration of an electronicdevice according to an embodiment.

Referring to FIG. 2, the electronic device 100 may include a camera 110,a communication interface 120, and a processor 130.

The camera 110 captures an image. Specifically, the camera 110 may abackground of the display device 200.

The camera 110 may be configured of an image sensor (not illustrated), alens (not illustrated), etc. and may process image frames such as imagedata acquired by the image sensor. Meanwhile, the image data acquired bythe camera 110 may include gradation values of pixels red (R), green(G), blue (B) (that is, Gray level, mainly 8 bits, and this may bepresented as Gray with 256 level).

The communication interface 120 may communicate with the display device200. The communication interface 120 may transmit/receive various datawith the display device.

In this regard, the communication interface 120 may performcommunication with the display device 200 through various communicationmethods. For example, the communication interface may use acommunication module to communicate with the display device 200according to a communication standard such as Bluetooth, Wi-Fi, or thelike.

The processor 130 may control the overall operation of the electronicdevice 100.

The processor 130 may capture the background of the display device 200through the camera 110. In this regard, the processor 130 may displaythe background of the display device 200 captured by the camera 110 as alive view image on a display (not illustrated) of the electronic device100, and when a command to capture is input, the processor may acquirethe live view image displayed on the display (not illustrated) tocapture a background image of the display device 200.

The background may include a background on which the display apparatus200 is located or the surrounding thereof.

Specifically, when the display device 200 is located at a specificlocation, the background may include the background area behind thedisplay device 200 which will be hidden when the display device 200 islocated, before the display device 200 is located at the specificlocation, or a background area around thereof. In addition, after thedisplay device 200 is located at the specific position, the backgroundmay include the background around the display device.

The processor 130 may divide the captured image into a plurality ofblocks. The sizes of the plurality of blocks may be identified based ona background pattern of the display device 200.

For example, the processor 130 may identify a size of the block based onat least one of the size, complexity, and type of the backgroundpattern, and divide the captured image into the plurality of blocksbased on the identified size.

Firstly, the processor 130 may identify the size of a block based on thesize of the background pattern, and divide the captured image into theplurality of blocks based on the identified size.

Specifically, the processor 130 may divide the captured image into theplurality of blocks such that each block has a size corresponding to thesize of the background pattern.

The larger the size of the background pattern is, the relatively largerthe size of the block may become, and the smaller the size of thebackground pattern is, the relatively smaller the size of the block maybecome.

For example, as illustrated in FIG. 3(a), when the background of thedisplay device 200 has a pattern 310 of fabric texture, and asillustrated in FIG. 3(b), the background of the display device 200 has apattern 320 of brickwork.

In this regard, the size of the pattern 320 of brickwork, that is, thesize of one brick, is larger than the size of the pattern 310 of thefabric texture, that is, the size of one object suggesting the fabrictexture.

The processor 130 may divide the captured background image into arelatively large block when the captured image has a pattern in whichbricks are laid as illustrated in FIG. 3(B), rather than a pattern offabric texture as illustrated in FIG. 3(A).

As such, the processor 130 may divide the captured image into arelatively large block as a size of the background pattern of thedisplay device 200 get larger, and divide the captured image into arelatively small block as the size of the background pattern of thedisplay device 200 gets smaller.

The processor 130 may identify the size of the background pattern basedon a frequency component of the captured image.

For example, the processor 130 may identify the frequency component ofthe captured image by acquiring a coefficient for each frequencycomponent of the captured image through a discrete cosine transform(DCT). However, this is only an embodiment and the processor 130 mayidentify the frequency component by converting an image signal into afrequency domain through various methods, in addition to the DCTconversion.

The processor 130 may identify that the background pattern has a firstsize when high-frequency components is relatively more thanlow-frequency components in the captured image (i.e., when the image ishigh frequency), and identify that the background pattern has a secondsize when the low-frequency components is relatively more than thehigh-frequency components (i.e., the image is low frequency). Here, thesecond size may be greater than the first size.

In other words, the processor 130 may identify that the size of thebackground pattern is small when the high-frequency components isrelatively more than the low-frequency components in the captured image,and identify that the size of the background pattern is large when thelow-frequency components is relatively more than the high-frequencycomponents in the captured image.

The processor 130 may identify the size of the block corresponding tothe size of the background pattern, and divide the captured image intothe plurality of blocks so that each block has the identified size.

The electronic device 100 may pre-store information on the size of theblock corresponding to the size of the background pattern.

For example, the electronic device 100 may store information about asize of a block corresponding to a small size pattern and informationabout a size of a block corresponding to a large size pattern.

Accordingly, the processor 130 may divide the captured image into theplurality of blocks based on the size of the pattern.

For example, FIG. 4(A) assumes that a histogram 410 with respect to acoefficient of each frequency component of the captured image.

As illustrated in FIG. 4(A), there are relatively more high-frequencycomponents than the low-frequency components in the captured image. Assuch, when the captured image is high frequency, the processor 130 mayidentify that the background pattern of the display device 200 is small,and as illustrated in FIG. 4(B), and divide the captured image into theplurality of blocks 420 such that each block has a size corresponding tothe size of pattern.

In addition, it is assumed that the histogram 430 of the coefficient ofeach frequency component of the captured image is as shown in FIG.

As illustrated in FIG. 4(C), there are relatively more low-frequencycomponents than high-frequency components in the captured image. Assuch, when the captured image is a low frequency, the processor 130 mayidentify that the background pattern of the display device 200 is large,and as illustrated in FIG. 4(D), and divide the captured image into theplurality of blocks 420 such that each block has a size corresponding tothe size of pattern.

Accordingly, the larger the pattern of the background of the displaydevice 200 is, the larger the size of the block may become, and thesmaller the pattern of the background of the display device 200 is, thesmaller the size of the block may become.

Meanwhile, the processor 130 may identify the size of the block based oncomplexity of the background, and divide the captured image into aplurality of blocks based on the identified complexity.

In other words, the processor 130 may divide the captured image into theplurality of blocks such that each block has a size corresponding to thecomplexity of the background.

In this regard, the higher the complexity of the background is, therelatively larger the block size may become, and the lower thecomplexity of the background is, the relatively smaller the block sizemay become.

For example, it is assumed that the background of the display device 200has various patterns 510 as illustrated in FIG. 5(A), the background ofthe display device 200 has a pattern 520 of narrow wooden boardsstanding side by side as illustrated in FIG. 5(B), and the background ofthe display device 200 has a pattern 530 in which square bricks are laidas shown in FIG. 5(C).

In this case, a background pattern may be complex in an order of thepattern 530 in which square bricks are laid, the pattern 520 of narrowwooden boards standing side by side, and the various patterns 510.

In this regard, the processor 130 may divide the captured backgroundimage into blocks of relatively larger sizes when the background has thepattern 520 of narrow wooden boards standing side by side as illustratedin FIG. 5(B) rather than the pattern 530 in which square bricks are laidas illustrated in FIG. 5(C), and the processor 130 may divide thecaptured background image into blocks of relatively larger sizes whenthe background has the various patterns 510 rather than the pattern 520of narrow wooden boards standing side by side.

As described above, as the complexity of the background pattern of thedisplay device 200 increases, the processor 130 may divide the capturedimage into blocks of relatively large size, and as the complexity of thebackground pattern of the display device 200 decreases, the processor130 may divide the captured image into blocks of relatively small size.

The processor 130 may identify the complexity of the background patternbased on a variance of the captured image.

To be specific, the processor 130 may calculate a variance of gradationvalue of each pixel of the captured image, and identify complexitycorresponding to the calculated variance among the plurality ofvariances as the complexity of the background pattern.

Information about the complexity corresponding to the variance may bepre-stored in the electronic device 100.

For example, the electronic device 100 may store information about whena variance is less than a, complexity corresponding thereto is low, andwhen a variance is more than a and less than b, complexity correspondingthereto is medium, and when a variance is more than or equal to b,complexity corresponding thereto is high (where a<b).

For example, as for FIG. 5, it is assumed that when the background imagehaving the pattern 510 as illustrated in FIG. 5(A) is captured, thevariance calculated from the captured background image is 2416, when thebackground image having the same pattern 520 as the FIG. 5(B) iscaptured, the variance calculated from the captured background image is977, and when the background image having the same pattern 530 as FIG.5(C) is captured, the variance calculated from the captured backgroundimage is 139.

In this case, the processor 130 may identify that complexity of thebackground image having the same pattern 510 is high as illustrated inFIG. 5(A), complexity of the background image having the same pattern520 as illustrated in FIG. 5(B) is medium, and complexity of thebackground image having the same pattern 530 as illustrated in FIG. 5(C)is low.

The processor 130 may identify the size of a block corresponding to thecomplexity of the background pattern, and divide the captured image intothe plurality of blocks so that each block has an identified size.

In this regard, the electronic device 100 may pre-store information onthe size of the block corresponding to the complexity of the backgroundpattern.

For example, the electronic device 100 may pre-store information on asize of the block corresponding to the pattern of low complexity,information on a size of the block corresponding to the pattern ofmedium complexity, and information on a size of the block correspondingto the pattern of high complexity.

Accordingly, the processor 130 may divide the captured image into theplurality of blocks based on the complexity of the pattern.

For example, complexity of the background having the same pattern 510 asillustrated in FIG. 5(A) is high. In this regard, the processor 130 maydivide the captured image into the plurality of blocks 610 such thateach block has a size corresponding to the size of the high complexityas illustrated in FIG. 6(A). In addition, complexity of the backgroundhaving the same pattern 520 as illustrated in FIG. 5(B) is medium. Inthis regard, the processor 130 may divide the captured image theplurality of blocks 620 such that each block has a size corresponding tothe size of the medium complexity as illustrated in FIG. 6(B). Inaddition, complexity of the background having the same pattern 530 asillustrated in FIG. 5(C) is low. In this regard, the processor 130 maydivide the captured image into the plurality of blocks 630 such thateach block has a size corresponding to the size of the low complexity asillustrated in FIG. 6(C).

Accordingly, the more complex the background pattern of the displaydevice 200 is, the larger the size of the block may become, and the lesscomplex the background pattern of the display device 200, the smallerthe size of the block may become.

Meanwhile, the processor 130 may identify the size of the block based onthe type of the background pattern, and divide the captured image intothe plurality of blocks based on the identified size.

Specifically, the processor 130 may divide the captured image into theplurality of blocks such that each block has a size corresponding to thetype of the background pattern.

In this regard, the processor 130 may identify the type of thebackground pattern by performing an image search with respect to thecaptured image through a web server (not illustrated).

To this end, a communication interface 120 may include a communicationmodule for connecting to the web server (not illustrated) through anetwork.

Specifically, the processor 130 may transmit an image captured throughthe communication interface 120 to the web server (not illustrated), andreceive a result of the image search from the web server (notillustrated) through the communication interface 120.

Accordingly, the processor 130 may identify the type of the backgroundpattern based on the result of the image search. The processor 130 mayidentify the size of the block corresponding to the type of thebackground pattern, and divide the captured image into the plurality ofblocks such that each block has the identified size.

To this end, the electronic device 100 may pre-store information on thesize of the block corresponding to the type of the background pattern,or the like.

For example, as illustrated in FIG. 7(A), when a brickwork is recognizedin the captured image 710 (720) as a result of the image search, theprocessor 130 may identify that the background pattern is a brick type,and divide the captured image 710 into the plurality of blocks 730 suchthat each block has a size corresponding to a brick type, as illustratedin FIG. 7(B).

In addition, as illustrated in FIG. 7(B), when a texture or a wallpaperis recognized in the captured image 740 (750) as a result of the imagesearch, the processor 130 may identify that the background pattern is afabric texture type, and divide the captured image 740 into theplurality of blocks 760 such that each block has a size corresponding tothe fabric texture type, as illustrated in FIG. 7(D).

As such, the processor 130 may divide the captured image into theplurality of blocks based on at least one of the size, complexity, andtype of the background pattern.

Thereafter, the processor 130 may compare gradation values of the eachof the plurality of blocks with a target gradation value to adjust thegradation values of the each of the plurality of blocks.

Specifically, the processor 130 may normalize the gradation values ofthe each of the plurality of blocks and compare average values of thenormalized respective gradation values with the normalized gradationvalues to calculate an adjustment value for the each of the plurality ofblocks, and adjust the gradation values of the each of the plurality ofblocks based on the calculated adjustment value.

Firstly, the processor 130 may identify a representative value of R, G,and B gradation values for the each of the plurality of blocks.

For example, the processor 130 may identify an average value of the R,G, and B gradation values of pixels of the respective blocks asrepresentative values of the R, G, and B gradation values of therespective blocks. However, this is only an embodiment, and theprocessor 130 may identify the smallest R, G, B gradation value or thelargest R, G, B gradation value among the R, G, B gradation values ofthe pixels of the respective blocks as representative values of the R,G, and B gradation values of the respective blocks.

The processor 130 may calculate an average value of the R, G, and Bgradation values for the plurality of blocks by using the R, G, and Brepresentative values for the respective blocks. In other words, theprocessor 130 may use the R, G, and B representative values for therespective blocks to calculate the average value of the R, G, and Bgradation values for an entire captured image.

For example, assuming that the representative values of the R, G, and Bgradation values of a first block are 221, 226, and 235, and therepresentative values of the R, G, and B gradation values of a secondblock are 232, 235, and 186, and all blocks are composed of these twoblocks.

In this case, the processor 130 may calculate average values of the R,G, and B gradation values of the entire block to 226.5 (=(221+232)/2),230.5 (=(226+235)/2), 210.5 (=(235+186)/2).

Thereafter, the processor 130 may normalize the representative value ofthe R, G, and B gradation values for the respective blocks and theaverage values of the R, G, and B gradation values for the the pluralityof blocks.

For example, the processor 130 may normalize representative values ofthe R, G, and B gradation values for the respective blocks and averagevalues of the R, G, and B gradation values for the the plurality ofblocks, through R/G and B/G.

In other words, assuming that the R, G, and B representative values ofthe first block are 221, 226, and 235, and the R, G, and Brepresentative values of the second block are 232, 235, and 186, and theR, G, and B averages values for the entire block are 226.5, 230.5, and210.5.

In this case, the processor 130 may normalize the R, G, and Brepresentative values for the first block to 0.978 (=R/G=(221/226)) and1.04 (=B/G=(235/226)). R, G, and the R, G, and B representative valuesfor the second block to 0.987 (=R/G=(232/235)), 0.791 (=B/G=(186/235)),and the R, G, and B average values for the entire block to 0.983(=R/G=(226.5/230.5)).

Thereafter, the processor 130 may compare the normalized gradationvalues for the respective blocks with the normalized average values forthe the plurality of blocks, identify the R, G, and B gradation values,which should be adjusted in the respective blocks, in order that thenormalized gradation values for the respective blocks become normalizedaverage values for the the plurality of blocks, and adjust the R, G, andB gradation values of the pixels of the respective blocks as much as theidentified value to adjust gradation values of the each of the pluralityof blocks.

For example, the processor 130 may identify the R, G, and B gradationvalues, which should be adjusted, in order that the normalized gradationvalues 0.978 and 1.04 for the first block become the normalized averagevalues 0.983 and 0.913 for the entire block, and adjust the R, G, and Bgradation values of the respective pixels of the first block as much asthe identified gradation values. In addition, the processor 130 mayidentify the R, G, and B gradation values, which should be adjusted, inorder that the normalized gradation values 0.987 and 0.791 for thesecond block become the normalized average values 0.983 and 0.913 forthe entire block., and adjust the R, G, and B gradation values of therespective pixels of the second block as much as the identifiedgradation values.

Based on the method described above, the processor 130 may adjust thegradation value of the captured image for the respective blocks.

Thereafter, the processor 130 may transmit the image with adjustedgradation values to the display device 200 through the communicationinterface 120.

In other words, the processor 130 may transmit the background image withadjusted gradation values to a ray device 10, and display the backgroundimage received from the electronic device 100.

As described above, according to various embodiments of the disclosure,a white balance of the captured image may be more precisely corrected inthat the gradation values of the captured background image are adjustedby the respective blocks.

In addition, the size of the block is identified based on the size,complexity and type of the background pattern, so that the R, G, Bgradation values of deviation, caused by the pattern, can be reducedwhen light shines on the pattern by an illumination, thereby minimizingan effect of the pattern and correcting the white balance.

FIG. 8 is a block diagram illustrating a detailed configuration of anelectronic device according to an embodiment.

Referring to FIG. 8, the electronic device 100 may include a camera 110,a communication interface 120, a processor 130, a sensor 140, a display150, an audio outputter 160, and a user inputter 170 and a memory 180.

Meanwhile, the elements of the electronic device 100 illustrated in FIG.8 is merely one of embodiments, and may not be necessarily limited tothe block diagram described above. Thus, one or more of the elements ofthe electronic device 100 illustrated in FIG. 8 may be omitted ormodified, or one or more elements may be added to the electronic device100.

Since the camera 110, the communication interface 120, and the processor130 have been described with reference to FIG. 2, detailed descriptionsthereof will be omitted.

The camera 110 captures an image. To this end, the camera 110 may becomposed of an image sensor 111, a lens 112, a flash 113, and the like,and may process an image frame such as a still image or a video acquiredby the image sensor.

The communication interface 120 may communicate with various types ofexternal devices according to various methods of communication.

For example, the communication interface 120 may include one or moremodules which enable wireless communication between the electronicdevice 100 and a wireless communication system, between the electronicdevice 100 and another electronic device 100, or between the electronicdevice 100 and an external server. In addition, the wirelesscommunication interface 120 may include one or more modules forconnecting the electronic device 100 to one or more networks.

The communication interface 120 may include at least one among abroadcast receiving chip 121, a wireless communication chip 122, and ashort range communication chip 123. The processor 130 may communicatewith an external server or various external devices using thecommunication interface 120.

The sensor 140 may include one or more sensors for sensing at least oneof information in the electronic device 100, environment information ofthe electronic device 100, and user information.

For example, the sensor 140 may include an infrared sensor (IR sensor)141, a laser sensor 142, a thermal sensor 143, and an illuminationsensor 144. The processor 130 may utilize information sensed by thesesensors.

The display 150 may display content. The content may include a stillimage such as a photo, a document, a web page, a broadcast program, amovie, and the like, and a video.

The display 150 may be implemented as various types of displays, such asa liquid crystal display (LCD), and the like.

In addition, the display 150 may be implemented as a touch screenincluding a layer structure in combination with a touch panel (notillustrated). The touch screen may have not only a display function, butalso a function to detect not only a touch input location and a touchedinput area but also a touch input pressure. Further, the touch screenmay have a function to detect a proximity touch as well as a real touch.

Accordingly, the display 150 may provide an input interface between theelectronic device 100 and the user, and may also provide an outputinterface between the electronic device 100 and the user.

The audio outputter 160 is a component that outputs various noticesounds or voice message as well as various audio. Specifically, theaudio outputter 160 may be implemented as a speaker, but this is merelyone of various embodiments of the disclosure. The audio outputter 160may be implemented as an output terminal capable of outputting audio.

The user inputter 170 may receive and transmit various user inputs tothe processor 130.

The user inputter 170 may include a touch panel (not illustrated), a(digital) pen sensor (not illustrated), or a key (not illustrated). Thetouch panel (not illustrated) may, for example, use at least one ofelectrostatic type, pressure sensitive type, infrared type, and anultraviolet type. The touch panel (not illustrated) may further includea control circuit. The touch panel (not illustrated) may further includea tactile layer to provide a tactile response to a user. The (digital)pen sensor (not illustrated), for example, may be part of a touch panelor include a separate detection sheet. The key (not illustrated), forexample, may include a physical button, an optical key, or a keypad.

The processor 130 may control the overall operation of the electronicdevice 100. For example, the processor 130 may control hardwarecomponents or software elements connected to the processor 130 bydriving an O/S or an application program and process or compute variousdata. Further, the processor 130 may load and process a command or datareceived from at least one of the other components to a volatile memoryand store diverse data in a non-volatile memory.

For this operation, the processor 130 may be realized a dedicatedprocessor for performing functions (e.g., embedded processor) or ageneric-purpose processor for performing functions by running one ormore software programs stored in a memory device (e.g., a CPU or anapplication processor).

The processor 130 may include a ROM 131, a RAM 132, a CPU 133, and a GPU134, and these may be connected to each other through a bus 135.

The CPU 133 may access a memory 165 and boot using the 0/S stored in thememory 165. The CPU 133 may also perform various operations by usingvarious types of programs, contents, data, and the like stored in thememory 165.

The ROM 131 may store a set of commands for system booting. If a turn-oncommand is input and the power is supplied, the CPU 133 copies the 0/Sstored in the memory 180 into the RAM 132 according to the commandstored in the ROM 131, and boots the system by executing the O/S. Whenthe booting is completed, the CPU 133 may copy the various programsstored in the memory 180 to the RAM 132, and perform various operationsby implementing the programs copied to the RAM 132.

Upon completion of the boot-up operation, the GPU 134 may generate ascreen including various objects, such as icons, images, text, or thelike.

The memory 180 may store various programs and data necessary for theoperation of the electronic device 100. The memory 180 may store commandor data received from the processor 130 or other components or generatedby the processor 130 or other components.

The memory 180 may be implemented as a non-volatile memory, a volatilememory, a flash memory, a hard disk drive (HDD), a solid state drive(SDD), and the like. The memory 180 may be accessed by the processor130, and perform readout, recording, correction, deletion, update, andthe like, on data by the processor 130.

Meanwhile, in the above-described embodiment, it has been described thatthe electronic device 100 captures the background of the display device200 and adjusts gradation values of the captured image.

However, in addition to the method above, the display device 200 mayadjust a gradation value of an image displayed or adjust brightness, andthis will be described in more detail below.

FIG. 9 is a block diagram illustrating a configuration of a displaydevice according to an embodiment.

As shown in FIG. 9, the display device 200 includes a sensor 210, acommunication interface 220, a display 230, and a processor 230.

The sensor 210 may receive light around the display device 200 anddetect a gradation value of the received light.

The sensor 210 may be disposed at a specific position of the displaydevice 200 by being composed of one sensor, or disposed at a positionspaced apart from each other on the display device 200 by being composedof a plurality of sensors.

The sensor may be implemented as various types of sensors (e.g.,illuminance sensor, color sensor, etc.) capable of detecting R, G, and Bgradation values of light, or detecting illuminance, color coordinates,and color temperature.

The communication interface 220 communicates with an external device.The communication interface 220 may transmit and receive various datawith the external device 200. The external device may refer to theelectronic device 100.

Specifically, the communication interface 220 may perform communicationwith the electronic device 100 through various communication methods.For example, the communication interface 220 may perform communicationwith the electronic device 100 according to a communication standardsuch as Bluetooth, Wi-Fi, etc. using a communication module.

The display 230 displays an image. In particular, the display 230 may becaptured by the electronic device 100 and display a background imagereceived from the electronic device 100 through the communicationinterface 220.

The display 230 may include a display panel (not illustrated) such as aliquid crystal display (LCD), a backlight (not illustrated), a paneldriver (not illustrated), and a light source controller (notillustrated).

The processor 240 controls overall operations of the display device 200.For example, the processor 240 may control hardware components orsoftware elements connected to the processor 240 by driving the O/S oran application program and process or compute various data. Further, theprocessor 240 may load and process a command or data received from atleast one of the other components to a volatile memory and store diversedata in a non-volatile memory.

For this operation, the processor 240 may be realized a dedicatedprocessor for performing functions (e.g., embedded processor) or ageneric-purpose processor for performing functions by running one ormore software programs stored in a memory device (e.g., a CPU or anapplication processor).

In this example, the processor 240 may receive a background imagecaptured by the electronic device 100.

The processor 240 may receive the R, G, and B gradation values of lightaround the display device 200 detected by the sensor 210, and use the R,G, and B gradation values to identify color coordinates, colortemperature and illuminance around the display device 200.

The processor 240 may adjust the gradation value of an image displayedon the display 230 based on the color coordinates or the colortemperature.

For this operation, the display device 200 may store information about agradation value (hereinafter, referred to as a corrected gradationvalue) corrected according to color coordinates or color temperaturearound the display device 200.

Accordingly, the processor 240 may correct the gradation value of theimage based on a pre-stored corrected gradation value, and display thecorrected image on the display 230.

For example, when it is identified that the light around the displaydevice 200 has a yellow-based color (e.g., when the color temperature is4000K), the processor 240 may control the panel driver (not illustrated)to increase the B gradation value among the R, G, and B gradation valuesof the background image based on the pre-stored corrected gradationvalue, and display the corrected background image to the display panel(not illustrated).

As another example, when it is identified that the light around thedisplay device 200 has a blue-based color (e.g., when the colortemperature is 6500K), the processor 240 may control the panel driver(not illustrated) to increase the R gradation value among the R, G, andB gradation values of the background image based on the pre-storedcorrected gradation value, and display the corrected background image tothe display panel (not illustrated).

As described above, according to an embodiment of the disclosure, thecolor coordinates or the color temperature of the image displayed on thedisplay 230 are adjusted in real time according to a color of an ambientlight source of the display device 200 to prevent the background imagedisplayed on the display device 200 from being displayed to the user ina different color by the ambient light source.

Meanwhile, the processor 240 may adjust luminance of the display 230based on the illuminance.

For this operation, the display device 200 may store information on theluminance of the display 230 according to the illuminance around thedisplay device 200.

Accordingly, the processor 240 may identify luminance of an image basedon the pre-stored luminance information, and display the image of theidentified luminance on the display 230.

For example, when an illuminance level is i, and luminance storedcorresponding thereof is 1, the processor 240 may control the lightsource controller (not illustrated) when the illuminance level aroundthe display device 200 is i, a background image may be displayed on thedisplay panel (not illustrated) with 1 luminance.

Accordingly, when surrounding of the display device 200 is bright, theprocessor 240 may display an image with high luminance.

As such, according to an embodiment of the disclosure, as a light of thedisplay device 200 is reflected by the ambient light source so that thescreen of the display device 200 is hardly visible to the user, it ispossible to prevent the background image displayed on the display device200 from being seen different from background around the backgroundimage to the user.

The display device 200 may include a plurality of sensors. The pluralityof sensors may be disposed at positions spaced apart from each other onthe display device 200. For example, sensors may be disposed on left andright sides of the display device 200, respectively.

The processor 240 may partially adjust the graduation value of the imageor partially adjust the luminance of the display 230 based on thegradation value detected by each sensor.

For example, it is assumed that a yellow-based light source exists onthe left side of the display device 200 according to sensing data of thesensor disposed on the left side of the display device 200, and ablue-based light source exists on the right side of the display device200 according to sensing data of the sensor disposed on the right sideof the display device 200.

The processor 240 may increase the B gradation value among the R, G, andB gradation values of the background image displayed on the left area ofthe display panel (not illustrated), and increase the R gradation valueamong the R, G, and B gradation values of the background image displayedon the right area of the display panel (not illustrated) to display acorrected background image on the display panel (not illustrated).

As another example, it is assumed that the illuminance on the left ofthe display device 200 is i₁, and the illuminance on the right of thedisplay device 200 is i₂ according to the sensing data of the sensordisposed on the left of the display device 200.

The processor 240 may control the light source controller (notillustrated) to display the background image displayed on the left areaof the display panel (not illustrated) as l₁, luminance corresponding tothe illuminance level i₁, and the background image displayed on theright area of the display panel (not illustrated) as l₂, luminancecorresponding to the illuminance level i₂, and display an image of whichluminance is adjusted on the display panel (not illustrated).

As such, according to an embodiment of the disclosure, the gradationvalue and the luminance of the image may be partially adjusted.

Meanwhile, as for another example, the processor 240 may detect adirection in which light shines on the display device 200 through asensor, and display content on the display 230 according to thedirection of the light.

For example, it is assumed that a sensor is disposed on each of upperleft and upper right sides of the display device 200. When the light isdetected by the sensor on the upper left side, the processor 240 mayidentify that a light source exists on the left of the display device200, and when the light is detected by the sensor on the upper rightside, the processor 240 may identify that the light source exists on theright of the display device.

In addition, the processor 240 may identify a direction of shadowdisplayed on the display 230 according to the direction of light, anddisplay shadow on a content based on the identified direction of theshadow.

For example, the processor 240 may display a shadow in the rightdirection of the content when the light source exists on the left of thedisplay device 200, and display a shadow in the left direction of thecontent when the light source exists on the left side of the displaydevice 200.

Meanwhile, as for another example, the processor 240 may identify acurrent time based on time information set in the display device 200 orreal time logic of the processor 240, or receive time information from anetwork to identify the current time.

In addition, when the current time is in the night time and an ambientilluminance detected by the sensor is less than or equal to a presetvalue, the processor 240 may cut off power supplied to the panel driver(not illustrated) and the light source controller (not illustrated).Accordingly, power consumption can be reduced.

Meanwhile, when the current time is in the daytime, the processor 240may identify whether ultraviolet light exists in the light detected bythe sensor, and when the ultraviolet light is detected, the processor240 may increase luminance of an image displayed on the display panel(not illustrated) to a specific value, thereby solving the problem thatthe screen is hard to be seen due to the external light.

As described above, according to various embodiments of the disclosure,when the display device 200 displays the background screen, it ispossible to resolve the difference that the user feels according toillumination environment, and thus maximize a transparent effect.

Meanwhile, the embodiment described above has described that theelectronic device 100 captures the background of the display device 200,divides the captured image into the plurality of blocks, adjusts thegradation value of the each of the plurality of blocks, and transmit theimage in which the gradation value is adjusted to the display device200.

However, this is merely an embodiment, and may be performed through thedisplay device 200.

That is, the electronic device 100 may capture the background of thedisplay device 200 and transmit the captured image to the display device200. The processor 240 of the display device 200 may divide the imagereceived from the electronic device 100 into the plurality of blocks,adjust the gradation values of the each of the plurality of blocks, anddisplay the image of which the gradation value is adjusted on thedisplay 230.

Meanwhile, the method of dividing the image into the plurality of blocksin the display device 200 and adjusting the gradation values of therespective blocks may be the same as the method performed by theelectronic device 100. The operation performed by components of theelectronic device 100 may be performed in components of the displaydevice 200 corresponding thereto (for example, an operation performed bythe processor 130 of the electronic device 100 may be performed by theprocessor 240 of the display device 200).

FIG. 10 is a flowchart illustrating an image correction method accordingto an embodiment.

The background of the display device is captured (S1010), and thecaptured image is divided into the plurality of blocks (S1020).

Thereafter, a target gradation value is compared with gradation valuesof the each of the plurality of blocks to adjust the gradation values ofthe each of the plurality of blocks (S1030), and an image, the gradationvalues of which have been adjusted, is transmitted to the display(S1040).

In this regard, sizes of the plurality of blocks may be identified basedon the background pattern of the display device.

Specifically, as for S1020, the block size may be identified based on atleast one of a size, complexity, and a type of the background pattern,and the captured image may be divided into the plurality of blocks onthe basis of the identified size.

Firstly, as for step S1020, the captured image is divided into theplurality of blocks such that each block has a size corresponding to thesize of the background pattern. The larger the size of the backgroundpattern is, the relatively larger the size of the block may become, andthe smaller the size of the background pattern is, the relativelysmaller the size of the block may become.

In this regard, based on the frequency component of the captured image,when there are relatively more high-frequency components than thelow-frequency components in the captured image, it may be identifiedthat the background pattern has the first size, and when there arerelatively more low-frequency components than the high-frequencycomponents in the captured image, it may be identified that thebackground pattern has the second size. The second size may be largerthan the first size.

Meanwhile, as for S1020, the captured image is divided into theplurality of blocks such that each block has a size corresponding to thecomplexity of the background. The higher the complexity of thebackground is, the relatively larger the size of the block, and thelower the complexity of the background is, the relatively smaller thecomplexity of the background may become.

A variance of the captured image may be calculated, and the capturedimage may be divided into the plurality of blocks based on thecomplexity corresponding to the variance among the plurality ofcomplexity.

In addition, as for S1020, the captured image may be divided into theplurality of blocks such that each block has a size corresponding to thetype of the background pattern.

In this regard, an image search may be performed regarding the imagecaptured through the web server to identify the type of the backgroundpattern.

Meanwhile, normalizing the gradation values of the plurality of blocks,comparing the normalized respective gradation values with the averagevalues of the normalized gradation values to calculate an adjustmentvalue for the each of the plurality of blocks, and the gradation valuesof the each of the plurality of blocks may be adjusted based on thecalculated adjustment values.

The specific method regarding the image correction has been described.

According to an embodiment, the various embodiments described above maybe implemented as software including instructions stored in amachine-readable storage media which is readable by a machine (e.g., acomputer). The device may include the electronic device according to thedisclosed embodiments, as a device which calls the stored instructionsfrom the storage media and which is operable according to the calledinstructions. When the instructions are executed by a processor, theprocessor may directory perform functions corresponding to theinstructions using other components or the functions may be performedunder a control of the processor. The instructions may include codegenerated or executed by a compiler or an interpreter. Themachine-readable storage media may be provided in a form of anon-transitory storage media. The ‘non-transitory’ means that thestorage media does not include a signal and is tangible, but does notdistinguish whether data is stored semi-permanently or temporarily inthe storage media.

In addition, according to an embodiment, the methods according tovarious embodiments described above may be provided as a part of acomputer program product. The computer program product may be tradedbetween a seller and a buyer. The computer program product may bedistributed in a form of the machine-readable storage media (e.g.,compact disc read only memory (CD-ROM) or distributed online through anapplication store (e.g., Play Store™). In a case of the onlinedistribution, at least a portion of the computer program product may beat least temporarily stored or provisionally generated on the storagemedia such as a manufacturer's server, the application store's server,or a memory in a relay server.

Further, each of the components (e.g., modules or programs) according tothe various embodiments described above may be composed of a singleentity or a plurality of entities, and some subcomponents of theabove-mentioned subcomponents may be omitted or the other subcomponentsmay be further included to the various embodiments. Generally, oradditionally, some components (e.g., modules or programs) may beintegrated into a single entity to perform the same or similar functionsperformed by each respective component prior to integration. Operationsperformed by a module, a program, or other component, according tovarious embodiments, may be sequential, parallel, or both, executediteratively or heuristically, or at least some operations may beperformed in a different order, omitted, or other operations may beadded.

INDUSTRIAL APPLICABILITY

-

SEQUENCE LISTING FREE TEXT

-

What is claimed is:
 1. An electronic device comprising: a camera; acommunication interface; and a processor configured to capture an imageof a background of a display device through the camera, to divide thecaptured image into a plurality of blocks, to compare a target gradationvalue with gradation values of the each of the plurality of blocks, toadjust the gradation values of the each of the plurality of blocks, andto transmit an image of which the gradation values are adjusted to thedisplay device through the communication interface, wherein the sizes ofthe plurality of blocks are identified based on a background pattern ofthe display device.
 2. The electronic device as claimed in claim 1,wherein the processor is configured to identify the sizes of the blockson the basis of at least one of a size, complexity, and a type of thebackground pattern and to divide the captured image into the pluralityof blocks on the basis of the identified size.
 3. The electronic deviceas claimed in claim 2, wherein the processor is configured to divide thecaptured image into the plurality of blocks such that each block has asize corresponding to the background pattern, and the larger the size ofthe background pattern is, the relatively larger the size of the blockbecomes, and the smaller the size of the background pattern is, therelatively smaller the size of the block becomes.
 4. The electronicdevice as claimed in claim 3, wherein the processor is configured, onthe basis of frequency components of the captured image, to identifythat the background pattern has a first size, based on high-frequencycomponents being relatively more than low-frequency components in thecaptured image, and to identify that the background pattern has a secondsize, based on low-frequency components being relatively more thanhigh-frequency components in the capture image, wherein the second sizeis larger than the first size.
 5. The electronic device as claimed inclaim 2, wherein the processor is configured to divide the capturedimage into the plurality of blocks such that each block has a sizecorresponding to complexity of the background, and the higher thecomplexity of the background is, the relatively larger the size of theblock becomes, and the lower the complexity of the background is, therelatively smaller the size of the block becomes.
 6. The electronicdevice as claimed in claim 5, wherein the processor is configured tocalculate a variance of the captured image and divide the capture imageinto the plurality of blocks on the basis of complexity corresponding tothe variance among a plurality of complexity.
 7. The electronic deviceas claimed in claim 2, wherein the processor is configured to divide thecaptured image into the plurality of blocks such that each block has asize corresponding to the background pattern.
 8. The electronic deviceas claimed in claim 7, wherein the processor is configured to identify atype of the background pattern by performing image search for thecaptured image through a web server.
 9. The electronic device as claimedin claim 1, wherein the processor is configured to normalize thegradation values of the each of the plurality of blocks, compare thenormalized respective gradation values and average values of thenormalized gradation values to calculate adjustment values of the eachof the plurality of blocks, and adjust the gradation values of the eachof the plurality of blocks on the basis of the calculated adjustmentvalues.
 10. An image correction method of an electronic devicecomprising: capturing an image of a background of a display device;dividing the captured image into a plurality of blocks; comparing atarget gradation value with gradation values of the each of theplurality of blocks and adjusting the gradation values of the each ofthe plurality of blocks; and transmitting an image of which thegradation values are adjusted to the display device, wherein the sizesof the a plurality of blocks are identified based on a backgroundpattern of the display device.
 11. The method as claimed in claim 10,wherein the dividing comprises identifying the sizes of the blocks onthe basis of at least one of a size, complexity, and a type of thebackground pattern and dividing the captured image into the plurality ofblocks on the basis of the identified size.
 12. The method as claimed inclaim 11, wherein the dividing comprises dividing the captured imageinto the plurality of blocks such that each block has a sizecorresponding to the background pattern, and the larger the size of thebackground pattern is, the relatively larger the size of the blockbecomes, and the smaller the size of the background pattern is, therelatively smaller the size of the block becomes.
 13. The method asclaimed in claim 12, wherein the dividing comprises, on the basis offrequency components of the captured image, identifying that thebackground pattern has a first size, based on high-frequency componentsbeing relatively more than low-frequency components in the capturedimage, and identifying that the background pattern has a second size,based on low-frequency components being relatively more thanhigh-frequency components in the captured image, wherein the second sizeis larger than the first size.
 14. The method as claimed in claim 11,wherein the dividing comprises dividing the captured image into theplurality of blocks such that each block has a size corresponding tocomplexity of the background, and the higher the complexity of thebackground is, the relatively larger the size of the block becomes, andthe lower the complexity of the background is, the relatively smallerthe size of the block becomes.
 15. The method as claimed in claim 14,wherein the dividing comprises calculating a variance of the capturedimage and dividing the captured image into the plurality of blocks onthe basis of complexity corresponding to the variance among a pluralityof complexity.